Tracking interactivity with a prerecorded media file

ABSTRACT

In a method for tracking interactivity with a prerecorded video file superimposed into a video, presentation instructions for displaying a prerecorded video file are displayed on a display device of a mobile electronic device, the presentation instructions including display conditions for displaying the prerecorded video file. A video of a scene is displayed on the display device of the mobile electronic device. Responsive to detecting at least one display condition of the display conditions, the prerecorded video file is displayed on the display device of the mobile electronic device, such that the video is partially obscured by the prerecorded video file. Responsive to the displaying the prerecorded video file, a display instance for the prerecorded video file is logged.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of, and claims priority to and thebenefit of, co-pending U.S. Provisional Patent Application No.62/633,528, filed on Feb. 21, 2018, entitled “TRACKING INTERACTIVITYWITH A PRERECORDED MEDIA FILE,” by Rabbat et al., having Attorney DocketNo. GFYCAT-008.PRO, and assigned to the assignee of the presentapplication, the disclosure of which is hereby incorporated herein byreference in its entirety.

BACKGROUND

Mobile electronic devices, such as those configured to be handheld orotherwise associated with a user, are employed in a wide variety ofapplications and environments. The ubiquity of devices such as mobilephones, smart phones, and other similar devices indicates the popularityand desire for these types of devices. Mobile electronic devices areincreasingly used to interact with the physical environment.

BRIEF DESCRIPTION OF DRAWINGS

The accompanying drawings, which are incorporated in and form a part ofthe Description of Embodiments, illustrate various embodiments of thesubject matter and, together with the Description of Embodiments, serveto explain principles of the subject matter discussed below. Unlessspecifically noted, the drawings referred to in this Brief Descriptionof Drawings should be understood as not being drawn to scale. Herein,like items are labeled with like item numbers.

FIG. 1 is a block diagram of an example mobile electronic device, inaccordance with various embodiments.

FIG. 2 illustrates an example network upon which embodiments describedherein may be implemented.

FIGS. 3A through 3E illustrate examples of a prerecorded video filesuperimposed over a video, in accordance with various embodiments.

FIGS. 4A through 4C illustrate examples of a prerecorded video filesuperimposed over a video positioned relative to an object within thevideo, in accordance with various embodiments.

FIGS. 5A through 5C illustrate examples of a user-generated prerecordedvideo file superimposed over a video positioned relative to an objectwithin the video, in accordance with various embodiments.

FIG. 6 illustrates a flow diagram of an example method for superimposinga prerecorded media file into a video, according to various embodiments.

FIGS. 7A and 7B illustrates a flow diagram of an example method forselecting of a prerecorded media file for superimposing into a video,according to various embodiments.

FIG. 8 illustrates an example network upon which other embodimentsdescribed herein may be implemented.

FIG. 9 illustrates a flow diagram of an example method for trackinginteractivity with a prerecorded media file, according to variousembodiments.

DESCRIPTION OF EMBODIMENTS

Reference will now be made in detail to various embodiments of thesubject matter, examples of which are illustrated in the accompanyingdrawings. While various embodiments are discussed herein, it will beunderstood that they are not intended to limit to these embodiments. Onthe contrary, the presented embodiments are intended to coveralternatives, modifications and equivalents, which may be includedwithin the spirit and scope the various embodiments as defined by theappended claims. Furthermore, in this Description of Embodiments,numerous specific details are set forth in order to provide a thoroughunderstanding of embodiments of the present subject matter. However,embodiments may be practiced without these specific details. In otherinstances, well known methods, procedures, components, and circuits havenot been described in detail as not to unnecessarily obscure aspects ofthe described embodiments.

Notation and Nomenclature

Some portions of the detailed descriptions which follow are presented interms of procedures, logic blocks, processing and other symbolicrepresentations of operations on data bits within a computer memory.These descriptions and representations are the means used by thoseskilled in the data processing arts to most effectively convey thesubstance of their work to others skilled in the art. In the presentapplication, a procedure, logic block, process, or the like, isconceived to be one or more self-consistent procedures or instructionsleading to a desired result. The procedures are those requiring physicalmanipulations of physical quantities. Usually, although not necessarily,these quantities take the form of electrical or magnetic signals capableof being stored, transferred, combined, compared, and otherwisemanipulated in an electronic device.

It should be borne in mind, however, that all of these and similar termsare to be associated with the appropriate physical quantities and aremerely convenient labels applied to these quantities. Unlessspecifically stated otherwise as apparent from the followingdiscussions, it is appreciated that throughout the description ofembodiments, discussions utilizing terms such as “displaying,”“receiving,” “superimposing,” “playing,” “capturing,” “determining,”“using,” “placing,” “selecting,” “initiating,” “logging,” or the like,refer to the actions and processes of an electronic device such as: aprocessor, a processor, an optical sensor, a sensor, a memory, a mobileelectronic device, a sensor processing unit, a sensor processor, or thelike, or a combination thereof. The electronic device manipulates andtransforms data represented as physical (electronic and/or magnetic)quantities within the electronic device's registers and memories intoother data similarly represented as physical quantities within theelectronic device's memories or registers or other such informationstorage, transmission, processing, or display components.

Embodiments described herein may be discussed in the general context ofprocessor-executable instructions residing on some form ofnon-transitory processor-readable medium, such as program modules,executed by one or more computers or other devices. Generally, programmodules include routines, programs, objects, components, datastructures, etc., that perform particular tasks or implement particularabstract data types. The functionality of the program modules may becombined or distributed as desired in various embodiments.

In the figures, a single block may be described as performing a functionor functions; however, in actual practice, the function or functionsperformed by that block may be performed in a single component or acrossmultiple components, and/or may be performed using hardware, usingsoftware, or using a combination of hardware and software. To clearlyillustrate this interchangeability of hardware and software, variousillustrative components, blocks, modules, circuits, and steps have beendescribed generally in terms of their functionality. Whether suchfunctionality is implemented as hardware or software depends upon theparticular application and design constraints imposed on the overallsystem. Skilled artisans may implement the described functionality invarying ways for each particular application, but such implementationdecisions should not be interpreted as causing a departure from thescope of the present disclosure. Also, the example mobile electronicdevice described herein may include components other than those shown,including well-known components.

The techniques described herein may be implemented in hardware,software, firmware, or any combination thereof, unless specificallydescribed as being implemented in a specific manner. Any featuresdescribed as modules or components may also be implemented together inan integrated logic device or separately as discrete but interoperablelogic devices. If implemented in software, the techniques may berealized at least in part by a non-transitory processor-readable storagemedium comprising instructions that, when executed, perform one or moreof the methods described herein. The non-transitory processor-readabledata storage medium may form part of a computer program product, whichmay include packaging materials.

The non-transitory processor-readable storage medium may comprise randomaccess memory (RAM) such as synchronous dynamic random access memory(SDRAM), read only memory (ROM), non-volatile random access memory(NVRAM), electrically erasable programmable read-only memory (EEPROM),FLASH memory, other known storage media, and the like. The techniquesadditionally, or alternatively, may be realized at least in part by aprocessor-readable communication medium that carries or communicatescode in the form of instructions or data structures and that can beaccessed, read, and/or executed by a computer or other processor.

The various illustrative logical blocks, modules, circuits andinstructions described in connection with the embodiments disclosedherein may be executed by one or more processors, such as one or moremotion processing units (MPUs), sensor processing units (SPUs),processor(s) or core(s) thereof, digital signal processors (DSPs),general purpose microprocessors, application specific integratedcircuits (ASICs), application specific instruction set processors(ASIPs), field programmable gate arrays (FPGAs), or other equivalentintegrated or discrete logic circuitry. The term “processor,” as usedherein may refer to any of the foregoing structures or any otherstructure suitable for implementation of the techniques describedherein. In addition, in some aspects, the functionality described hereinmay be provided within dedicated software modules or hardware modulesconfigured as described herein. Also, the techniques could be fullyimplemented in one or more circuits or logic elements. A general purposeprocessor may be a microprocessor, but in the alternative, the processormay be any conventional processor, controller, microcontroller, or statemachine. A processor may also be implemented as a combination ofcomputing devices, e.g., a combination of an SPU/MPU and amicroprocessor, a plurality of microprocessors, one or moremicroprocessors in conjunction with an SPU core, MPU core, or any othersuch configuration.

Overview of Discussion

Discussion begins with a description of an example computer system uponwhich embodiments of the present invention may be implemented. Examplesof superimposing a prerecorded media file into a video are thendescribed. Examples of selection of a prerecorded media file forsuperimposing into a video are then described. Example operations ofsuperimposing a prerecorded media file into a video and selection of aprerecorded media file for superimposing into a video are thendescribed. Examples of tracking interactivity with a prerecorded mediafile superimposed into a video are then described.

In accordance with various embodiments, methods for superimposing aprerecorded media file into a video are provided. A video of a scene isdisplayed on a display device of a mobile electronic device. Aprerecorded video file to render on the display device is received. Theprerecorded video file is superimposed over the video, such that thevideo is partially obscured by the prerecorded video file. Theprerecorded video file is played while displaying the video, such thatthe prerecorded video file and a non-obscured portion of the video arerendered simultaneously. In some embodiments, live video is captured ata video capture device of the mobile electronic device, such that thevideo displayed on the display device is the live video.

Mobile electronic devices, in accordance with some describedembodiments, include cameras and display screens (e.g., displays ordisplay devices). These mobile electronic devices are capable forshowing a user a live video of a scene the user is looking at (e.g.,directing the camera towards). Embodiments described herein provide forthe superimposing of prerecorded video files over the live video, suchthat the prerecorded video file displays content over the live video.This content can be related to an object within the scene (e.g., aparticular location, an identified symbol or logo, an identified word orphrase, etc.) In some embodiments, the superimposed prerecorded videocontent can appear as if it is part of the geography of the scene,integrating directly into the scene.

Embodiments described herein pertain to superimposing a prerecordedmedia file into a video. In one embodiment, the prerecorded media fileis a Graphics Interchange Format (GIF) file. While embodiments describedherein pertain to GIF files, it should be appreciated that other typesof prerecorded media files, such as other types of video files, can beused herein. Moreover, it should be appreciated that any type of mediafile format can be used in accordance with the described embodiments,including but not limited to GIF, WebM, WebP, MPEG-4 (MP4), AnimatedPortable Network Graphics (APNG), Motion JPEG, Flash video (FLV),Windows Media video, M4V, etc. It should be appreciated that theprerecorded media file can be looped (e.g., via a HTML 5 video elementor Flash video element) to automatically repeat.

In some embodiments, the prerecorded video file is selectable forinteractivity. For example, a user may interact with a prerecorded videofile (e.g., touching it via a touch screen) to execute an action. Forexample, a user interaction with a prerecorded video file may open up anInternet browser application and navigate to a particular website. Inother examples, a user interaction with a prerecorded video file mayshow content (e.g., a coupon or product information). In someembodiments, the prerecorded video file may be transformed as a resultof a user interaction (e.g., the prerecorded video file can be resizedor moved within a display).

In some embodiments, depth information of the scene captured within thevideo is determined. The depth information defines distances of pointsof the scene from a video capture device, such that the depthinformation comprises relative depths of the points of the scene basedon a position of the video capture device.

In one embodiment, the prerecorded video file is placed over the videoat a particular distance from the position of the video capture devicebased on the depth information. In another embodiment, the prerecordedvideo file is placed over the video at the distances that the points ofthe scene are from the video capture device based on the depthinformation, such that the prerecorded video file is visually integratedinto the scene. In one embodiment, the depth information includessurface contours of objects within the scene, such that the prerecordedvideo file is placed over the video following the surface contours ofthe objects. In another embodiment, a flat region of the scene withinthe video is identified based on the depth information. The prerecordedvideo file is placed over the video at the flat region of the scene,such that the prerecorded video file is visually integrated into thescene at the flat region.

In accordance with some embodiments, methods for selecting of aprerecorded media file for superimposing into a video are provided. Avideo of a scene is displayed on a display device of a mobile electronicdevice. A location of the scene is determined. A prerecorded video fileis selected based at least in part on the location. The prerecordedvideo file is superimposed over the video, such that the video ispartially obscured by the prerecorded video file. In one embodiment, theprerecorded video file is an advertisement related to the location. Inanother embodiment, the prerecorded video file is a user-generated videofile selected from a library of media files. The prerecorded video fileis played while displaying the video, such that the prerecorded videofile and a non-obscured portion of the video are renderedsimultaneously. In some embodiments, live video is captured at a videocapture device of the mobile electronic device, such that the videodisplayed on the display device is the live video.

The selection of a prerecorded video file to superimpose over a scenemay be based on a location of the scene. For example, where the scene isa physical place of business (e.g., a restaurant or a clothing store), aprerecorded video file related to that physical place may besuperimposed over the video of the scene, to enhance the user'sexperience of viewing the scene. For instance, the prerecorded videofile may be an advertisement for the place of business, a coupon for useupon purchasing an item from the business, or any other informationchosen by the proprietor of the business (e.g., information on thebenefits of a product sold by the business.

In another example, the prerecorded video file that is selected forsuperimposing is user-generated content. For example, a user of themobile electronic device can subscribe to prerecorded video file feedsor can be linked to prerecorded video files via a social network friendconnection. Upon identifying a location, a prerecorded video file can besuperimposed over the scene to enhance the appearance of the scene. Forexample, a prerecorded video file can appear to decorate or adorn astructure in a neighborhood (e.g., a wall, a billboard, a skyline,etc.). In another example, a prerecorded video file may be superimposedover or around an identified symbol.

In one embodiment, the location of the scene is determined using anavigation satellite system sensor (e.g., Global Positioning System(GPS) sensor) of the mobile electronic device. In another embodiment,the location of the scene is determined using a third party application(e.g., Foursquare, Yelp). In another embodiment, symbol recognition isperformed on the video to identify at least one symbol (e.g., logo,word, phrase, etc.) A location database is searched for the symbol todetermine the location. In some embodiment, the prerecorded video fileis placed over the video without obscuring the at least one symbol.

Embodiments described herein provide methods for superimposing aprerecorded media file into a video and selection of a prerecorded mediafile for superimposing into a video. The described embodiments providevisual enhancements to scenes within the video, providing a richer anddeeper user experience. By superimposing a prerecorded video file over alive video, the scene captured by the live video is made fuller,providing enriched user interaction with their surroundings, promotinginteraction with the object of a scene (e.g., item for purchase, placeof business, etc.) beyond the confines of the real-world view of thelocation captured by the scene.

Example Mobile Electronic Device

Turning now to the figures, FIG. 1 is a block diagram of an examplemobile electronic device 100. As will be appreciated, mobile electronicdevice 100 may be implemented as a device or apparatus, such as ahandheld mobile electronic device, that can be moved in space by a humanuser. That is, mobile electronic device 100 is small enough to be heldin the hand of a human user, in various embodiments. For example, such amobile electronic device may be, without limitation, a mobile telephonephone (e.g., smartphone, cellular phone, a cordless phone running on alocal network, or any other cordless telephone handset), a wiredtelephone (e.g., a phone attached by a wire), a personal digitalassistant (PDA), a video game player, video game controller, anavigation device, an activity or fitness tracker device (e.g.,bracelet, clip, band, or pendant), a smart watch or other wearabledevice, a mobile internet device (MID), a personal navigation device(PND), a digital still camera, a digital video camera, a portable musicplayer, a portable video player, a portable multi-media player, a remotecontrol, a headset, or a combination of one or more of these devices.

As depicted in FIG. 1, mobile electronic device 100 may include aprocessor 110, a bus 120, a memory 130, at least one optical sensor 135,and a display 140. Some embodiments of mobile electronic device 100 mayfurther include one or more of an interface 150, a transceiver 160 (alldepicted in dashed lines), a location sensor 170, and/or othercomponents. In various embodiments, electrical power for mobileelectronic device 100 is provided by a mobile power source such as abattery, when not being actively charged.

Processor 110 can be one or more microprocessors, central processingunits (CPUs), DSPs, general purpose microprocessors, ASICs, ASIPs, FPGAsor other processors which run software programs or applications, whichmay be stored in memory 130, associated with the functions andcapabilities of mobile electronic device 100.

Bus 120 may be any suitable bus or interface to include, withoutlimitation, a peripheral component interconnect express (PCIe) bus, auniversal serial bus (USB), a universal asynchronousreceiver/transmitter (UART) serial bus, a suitable advancedmicrocontroller bus architecture (AMBA) interface, an Inter-IntegratedCircuit (I2C) bus, a serial digital input output (SDIO) bus, a serialperipheral interface (SPI) or other equivalent. In the embodiment shown,processor 110, memory 130, display 140, interface 150, transceiver 160,and other components of mobile electronic device 100 may be coupledcommunicatively through bus 120 in order to exchange commands and data.Depending on the architecture, different bus configurations may beemployed as desired. For example, additional buses may be used to couplethe various components of mobile electronic device 100, such as by usinga dedicated bus between processor 110 and memory 130.

Memory 130 can be any suitable type of memory, including but not limitedto electronic memory (e.g., read only memory (ROM), random accessmemory, or other electronic memory), hard disk, optical disk, or somecombination thereof. Multiple layers of software can be stored in memory130 for use with/operation upon processor 110. For example, an operatingsystem layer can be provided for mobile electronic device 100 to controland manage system resources in real time, enable functions ofapplication software and other layers, and interface applicationprograms with other software and functions of mobile electronic device100. Similarly, a user experience system layer may operate upon or befacilitated by the operating system. The user experience system maycomprise one or more software application programs such as menunavigation software, games, device function control, gesturerecognition, image processing or adjusting, voice recognition,navigation software, communications software (such as telephony orwireless local area network (WLAN) software), and/or any of a widevariety of other software and functional interfaces for interaction withthe user can be provided. In some embodiments, multiple differentapplications can be provided on a single mobile electronic device 100,and in some of those embodiments, multiple applications can runsimultaneously as part of the user experience system. In someembodiments, the user experience system, operating system, and/or theprocessor 110 may operate in a low-power mode (e.g., a sleep mode) wherevery few instructions are processed. Such a low-power mode may utilizeonly a small fraction of the processing power of a full-power mode(e.g., an awake mode) of the processor 110.

Optical sensor 135 may comprise, without limitation: a video capturedevice, a camera, and infrared camera, or other type of optical sensorfor capturing a video of a person, an object, and/or a scene. It shouldbe appreciated that mobile electronic device 100 may include more thanone optical sensor. In one example, optical sensor 135 is a back-sideoptical sensor (e.g., rear-facing camera) and optical sensor 138 is afront-side optical sensor (e.g., front-facing camera).

Display 140, may be a liquid crystal device, (organic) light emittingdiode device, or other display device suitable for creating and visiblydepicting graphic images and/or alphanumeric characters recognizable toa user. Display 140 may be configured to output images viewable by theuser and may additionally or alternatively function as a viewfinder forcamera. Display 140 is configured to display video captured at opticalsensor 135 (and optical sensor 138, when included).

Interface 150, when included, can be any of a variety of differentdevices providing input and/or output to a user, such as audio speakers,touch screen integrated with display 140, real or virtual buttons,joystick, slider, knob, printer, scanner, computer network I/O device,other connected peripherals and the like.

Transceiver 160, when included, may be one or more of a wired orwireless transceiver which facilitates receipt of data at mobileelectronic device 100 from an external transmission source andtransmission of data from mobile electronic device 100 to an externalrecipient. By way of example, and not of limitation, in variousembodiments, transceiver 160 comprises one or more of: a cellulartransceiver, a wireless local area network transceiver (e.g., atransceiver compliant with one or more Institute of Electrical andElectronics Engineers (IEEE) 802.11 specifications for wireless localarea network communication), a wireless personal area networktransceiver (e.g., a transceiver compliant with one or more IEEE 802.15specifications for wireless personal area network communication), and awired a serial transceiver (e.g., a universal serial bus for wiredcommunication).

Location sensor 170, when included, may be a sensor for determining alocation of mobile electronic device 100, such as a navigation satellitesystem sensor such as a Global Navigation Satellite System (GNSS)receiver (e.g., Global Positioning System (GPS) sensor) a and/or othercomponent configured to determine the location of mobile electronicdevice 100 from external radio signals. It is noted that thefunctionality of location sensor 170 may be implemented by processor110.

Example Network and System for Superimposing Prerecorded Video Filesinto a Video

FIG. 2 illustrates an example communication network 240 upon whichembodiments described herein may be implemented. FIG. 2 illustrateselectronic device 210 and remote computer system 230 which arecommunicatively coupled via network 240. It should be appreciated thatelectronic device 210 may be implemented as a mobile electronic device100, and/or include any combination of the components of mobileelectronic device 100.

Electronic device 210 is capable of displaying video of scene 250 (e.g.,at display 140). In one embodiment, scene 250 is captured at a videocapture device (e.g., optical sensor 135-1) of electronic device 210.Electronic device 210 may be associated with a particular user.

In one embodiment, remote computer system 230 is a server including amedia file library 232. A media file can be any type of file that can berendered on an electronic device 210 (e.g., a prerecorded video file).It should be appreciated that any type of media file format can be usedin accordance with the described embodiments, including but not limitedto Graphics Interchange Format (GIF), WebM, WebP, MPEG-4 (MP4), AnimatedPortable Network Graphics (APNG), Motion JPEG, Flash video (FLV),Windows Media video, M4V, etc. It should be appreciated that theprerecorded media file can be looped (e.g., via a HTML 5 video elementor Flash video element) to automatically repeat.

In some embodiments, electronic device 210 is capable of accessing mediafile library 232 for retrieving a prerecorded media file. It should beappreciated that electronic device 210 may include media file library232, or that media file library 232 may be distributed across bothelectronic device 210 and remote computer system 230. For example, asubset of media files of media file library 232 may be maintained withinmemory of electronic device 210 (e.g., frequently used media files) foraccess that does not require communication over network 240.

In some embodiments, a prerecorded video file for superimposing over avideo is selected based at least in part on the location of the scene.For example, where scene 250 is a physical place of business (e.g., acoffee shop), a prerecorded video file related to that business may besuperimposed over the video of the scene, to enhance the user'sexperience of viewing the scene. For instance, the prerecorded videofile may be an advertisement for the place of business, a coupon for useupon purchasing an item from the business, or any other informationchosen by the proprietor of the business (e.g., information on thebenefits of a product sold by the business.

In some embodiments, the prerecorded video file is selectable forinteractivity. For example, a user may interact with a prerecorded videofile (e.g., touching it via a touch screen) to execute an action. Forexample, a user interaction with a prerecorded video file may open up anInternet browser application and navigate to a particular website. Inother examples, a user interaction with a prerecorded video file mayshow content (e.g., a coupon or product information). In someembodiments, the prerecorded video file may be transformed as a resultof a user interaction (e.g., the prerecorded video file can be resizedor moved within a display).

In one embodiment, the location of the scene is determined using alocation sensor (e.g., location sensor 170) of the mobile electronicdevice. In another embodiment, the location of the scene is determinedusing a third party application (e.g., Foursquare, Yelp). In anotherembodiment, symbol recognition is performed on the video to identify atleast one symbol (e.g., logo, word, phrase, etc.) With reference to FIG.2, scene 250 depicts an exterior of a business including a sign 260.Sign 260 may include a text name 262 (e.g., ABC Coffee Shop) and/or alogo 264. In one embodiment, symbol recognition is performed on textname 262 and/or a logo 264. A location database 234 of locationsassociated with identified symbols is searched for the symbol todetermine the location. It should be appreciated that the location mayrefer to a name and/or type of business rather than a specific instanceof a business (e.g., the location is a Starbucks coffee shop, ratherthan one specific Starbucks).

Embodiments described herein provide a prerecorded video filesuperimposed over a video, wherein the prerecorded video file andnon-obscured portion of the video are rendered simultaneously. FIGS. 3Athrough 3E illustrate examples of a prerecorded video file superimposedover a video, in accordance with various embodiments.

FIG. 3A illustrates an example mobile electronic device 300 having adisplay 302. As illustrated, display 302 is displaying video 304, wherevideo 304 depicts a scene of a business location (e.g., a Starbuckscoffee shop). In one embodiment, video 304 is a live video captured by avideo capture device (e.g., a rear-facing camera) of mobile electronicdevice 300. It should be appreciated that in accordance with otherembodiments, video 304 may be captured at another mobile electronicdevice, and transmitted to mobile electronic device 300 for rendering ondisplay 302, either in real-time (e.g., live) or time delayed.

In one embodiment, video 304 is rendered via a camera application ofmobile electronic device 300. In another embodiment, video 304 isrendered via an application of mobile electronic device 300 forsuperimposing a prerecorded video file over video 304. It should beappreciated that the superimposing of a prerecorded video file over avideo may be performed at a standalone application of mobile electronicdevice 300 or via a plugin/modification of an existing application ofmobile electronic device 300. In one embodiment, the superimposing of aprerecorded video file over video 304 is executed responsive to a userinteraction with control element 308. Control element 308 is operable toactivate/deactivate the superimposing of a prerecorded video file overvideo 304.

Mobile electronic device 300 also plays prerecorded video file 306 overvideo 304. Prerecorded video file 306, as illustrated, includes subjectmatter related to the business location illustrated in video 304. Insome embodiments, prerecorded video file 306 is selected based on thelocation of the scene of video 304. For example, since video 304 is avideo of a Starbucks coffee shop, prerecorded video file 306 depictspouring and preparation of a cup of coffee, so as to entice the user ofmobile electronic device 300 to visit the coffee shop. It should beappreciated that in some embodiments, prerecorded video file 306 ispositioned within display 302 so as to not obscure important visualelements of video 304 (e.g., the name of the business location, the logoof the business location, etc.)

In one embodiment, static element 310 is also rendered on display 302.Static element 310 is an element for visually indicating thatprerecorded video file 306 is virtually held in place in space. Forexample, as illustrated, static element 310 depicts a piece of tape,indicating that prerecorded video file 306 is virtually fixed in spacevia static element 310. Other examples of static element 310 include apicture frame, a staple, an easel, etc.

FIG. 3B illustrates an example mobile electronic device 320 having adisplay 322. As illustrated, display 322 is displaying video 324, wherevideo 324 depicts a scene of a business location (e.g., an athleticclothing store). In one embodiment, video 324 is a live video capturedby a video capture device (e.g., a rear-facing camera) of mobileelectronic device 320. It should be appreciated that in accordance withother embodiments, video 324 may be captured at another mobileelectronic device, and transmitted to mobile electronic device 320 forrendering on display 322, either in real-time (e.g., live) or timedelayed.

In one embodiment, video 324 is rendered via a camera application ofmobile electronic device 320. In another embodiment, video 324 isrendered via an application of mobile electronic device 320 forsuperimposing a prerecorded video file over video 324. It should beappreciated that the superimposing of a prerecorded video file over avideo may be performed at a standalone application of mobile electronicdevice 320 or via a plugin/modification of an existing application ofmobile electronic device 320. In one embodiment, the superimposing of aprerecorded video file over video 324 is executed responsive to a userinteraction with control element 328. Control element 328 is operable toactivate/deactivate the superimposing of a prerecorded video file overvideo 324.

Mobile electronic device 320 also plays prerecorded video file 326 overvideo 324. Prerecorded video file 326, as illustrated, includes subjectmatter related to the business location illustrated in video 324. Insome embodiments, prerecorded video file 326 is selected based on thelocation of the scene of video 324. For example, since video 324 is avideo of an athletic clothing store, prerecorded video file 326 depictsa person running using athletic clothing and shoes available for sale atthe athletic clothing store, so as to entice the user of mobileelectronic device 320 to purchase clothing and/or shoes. It should beappreciated that in some embodiments, prerecorded video file 326 ispositioned within display 322 so as to not obscure important visualelements of video 324 (e.g., the name of the business location, the logoof the business location, the depicted clothing and/or shoes, etc.)

FIG. 3C illustrates an example mobile electronic device 340 having adisplay 342. As illustrated, display 342 is displaying video 344, wherevideo 344 depicts a scene of a business location (e.g., a Sephoracosmetics shop). In one embodiment, video 344 is a live video capturedby a video capture device (e.g., a rear-facing camera) of mobileelectronic device 340. It should be appreciated that in accordance withother embodiments, video 344 may be captured at another mobileelectronic device, and transmitted to mobile electronic device 340 forrendering on display 342, either in real-time (e.g., live) or timedelayed.

In one embodiment, video 344 is rendered via a camera application ofmobile electronic device 340. In another embodiment, video 344 isrendered via an application of mobile electronic device 340 forsuperimposing a prerecorded video file over video 344. It should beappreciated that the superimposing of a prerecorded video file over avideo may be performed at a standalone application of mobile electronicdevice 340 or via a plugin/modification of an existing application ofmobile electronic device 340. In one embodiment, the superimposing of aprerecorded video file over video 344 is executed responsive to a userinteraction with control element 350. Control element 350 is operable toactivate/deactivate the superimposing of a prerecorded video file overvideo 344.

Mobile electronic device 340 also plays prerecorded video files 346 and348 over video 344. Prerecorded video files 346 and 348, as illustrated,include subject matter related to the business location illustrated invideo 344. In some embodiments, prerecorded video files 346 and 348 areselected based on the location of the scene of video 344. For example,since video 344 is a video of a cosmetics shop, prerecorded video file346 depicts cosmetics in user and prerecorded video file 348 illustratesa video of the logo/name of the cosmetics shop, so as to entice the userof mobile electronic device 340 to visit the coffee shop. It should beappreciated that in some embodiments, prerecorded video files 346 and348 are positioned within display 342 so as to not obscure importantvisual elements of video 344 (e.g., the name of the business location,the logo of the business location, products available for purchase,etc.)

FIG. 3D illustrates an example mobile electronic device 360 having adisplay 362. As illustrated, display 362 is displaying video 364, wherevideo 364 depicts a scene of a business location (e.g., a movietheater). In one embodiment, video 364 is a live video captured by avideo capture device (e.g., a rear-facing camera) of mobile electronicdevice 360. It should be appreciated that in accordance with otherembodiments, video 364 may be captured at another mobile electronicdevice, and transmitted to mobile electronic device 360 for rendering ondisplay 362, either in real-time (e.g., live) or time delayed.

In one embodiment, video 364 is rendered via a camera application ofmobile electronic device 360. In another embodiment, video 364 isrendered via an application of mobile electronic device 360 forsuperimposing a prerecorded video file over video 364. It should beappreciated that the superimposing of a prerecorded video file over avideo may be performed at a standalone application of mobile electronicdevice 360 or via a plugin/modification of an existing application ofmobile electronic device 360. In one embodiment, the superimposing of aprerecorded video file over video 364 is executed responsive to a userinteraction with control element 372. Control element 372 is operable toactivate/deactivate the superimposing of a prerecorded video file overvideo 364.

Mobile electronic device 360 also plays prerecorded video files 366,368, and 370 over video 364. Prerecorded video files 366, 368, and 370as illustrated, include subject matter related to the business locationillustrated in video 364. In some embodiments, prerecorded video files366, 368, and 370 are selected based on the location of the scene ofvideo 364. For example, since video 364 is a video of a cosmetics shop,prerecorded video files 366, 368, and 370 illustrate videos of moviepreviews for movies currently showing at the movie theater, so as toentice the user of mobile electronic device 360 to see a movie at themovie theater and to guide the user in the selection of a particularmovie. It should be appreciated that in some embodiments, prerecordedvideo files 366, 368, and 370 are positioned within display 362 so as tonot obscure important visual elements of video 364 (e.g., the name ofthe business location, the logo of the business location, etc.)

FIG. 3E illustrates an example mobile electronic device 380 having adisplay 382. As illustrated, display 382 is displaying video 384, wherevideo 384 depicts a scene of a sidewalk (e.g., a sidewalk near or infront of a business). In one embodiment, video 384 is a live videocaptured by a video capture device (e.g., a rear-facing camera) ofmobile electronic device 380. It should be appreciated that inaccordance with other embodiments, video 384 may be captured at anothermobile electronic device, and transmitted to mobile electronic device380 for rendering on display 382, either in real-time (e.g., live) ortime delayed.

In one embodiment, video 384 is rendered via a camera application ofmobile electronic device 380. In another embodiment, video 384 isrendered via an application of mobile electronic device 380 forsuperimposing a prerecorded video file over video 384. It should beappreciated that the superimposing of a prerecorded video file over avideo may be performed at a standalone application of mobile electronicdevice 380 or via a plugin/modification of an existing application ofmobile electronic device 380. In one embodiment, the superimposing of aprerecorded video file over video 384 is executed responsive to a userinteraction with control element 388. Control element 388 is operable toactivate/deactivate the superimposing of a prerecorded video file overvideo 384.

Mobile electronic device 380 also plays prerecorded video file 386 overvideo 384. Prerecorded video file 386, as illustrated, includes subjectmatter related to a business near the location of the sidewalkillustrated in video 384. In some embodiments, prerecorded video file386 is selected based on the location of the scene of video 384. Forexample, video 384 is a video of a sidewalk near or in front of aStarbucks coffee shop. Prerecorded video file 386 depicts a video of abeverage available for purchase at the Starbucks coffee shop and acoupon for that beverage, so as to entice the user of mobile electronicdevice 380 to visit the coffee shop. It should be appreciated that insome embodiments, prerecorded video file 386 is positioned withindisplay 382 so as to not obscure important visual elements of video 384(e.g., the name of the business location, the logo of the businesslocation, traffic signs, etc.)

In one embodiment, static elements 390 and 392 are also rendered ondisplay 382. Static elements 390 and 392 are element for visuallyindicating that prerecorded video file 386 is virtually held in place inspace. For example, as illustrated, static elements 390 and 392 depictpieces of tape, indicating that prerecorded video file 386 is virtuallyfixed in space via static elements 390 and 392. Other examples of staticelements 390 and 392 include a picture frame, a staple, an easel, etc.

In some embodiments, depth information of the scene captured within thevideos is determined. The depth information defines distances of pointsof the scene from a video capture device, such that the depthinformation comprises relative depths of the points of the scene basedon a position of the video capture device. In accordance with variousembodiments, the depth information can be determined via the imagecapture device (e.g., a camera such as an infrared camera), or via asoftware development kit (SDK), such as Google's ARCore or Apple'sARKit.

In various embodiments, the prerecorded video file is placed over thevideo at a particular distance from the position of the video capturedevice based on the depth information. For example, as illustrated inFIGS. 3A through 3E, the prerecorded video files are positioned withinthe display so as to appear at some distance from the user (e.g., twofeet) and between the objects of the scene and the user. In suchembodiments, the prerecorded videos may appear to float within the scenebetween objects of the scene and the user.

In another embodiment, the prerecorded video file is placed over thevideo at the distances that the points of the scene are from the videocapture device based on the depth information, such that the prerecordedvideo file is visually integrated into the scene. In one embodiment, thedepth information includes surface contours of objects within the scene,such that the prerecorded video file is placed over the video followingthe surface contours of the objects. In another embodiment, a flatregion of the scene within the video is identified based on the depthinformation. The prerecorded video file is placed over the video at theflat region of the scene, such that the prerecorded video file isvisually integrated into the scene at the flat region.

FIGS. 4A through 4C illustrate examples of a prerecorded video filesuperimposed over a video positioned relative to an object within thevideo, in accordance with various embodiments. FIG. 4A illustrates anexample mobile electronic device 400 having a display 402. Asillustrated, display 402 is displaying video 404, where video 404depicts a scene of a business location (e.g., a local coffee shop). Inone embodiment, video 404 is a live video captured by a video capturedevice (e.g., a rear-facing camera) of mobile electronic device 400. Itshould be appreciated that in accordance with other embodiments, video404 may be captured at another mobile electronic device, and transmittedto mobile electronic device 400 for rendering on display 402, either inreal-time (e.g., live) or time delayed.

In one embodiment, video 404 is rendered via a camera application ofmobile electronic device 400. In another embodiment, video 404 isrendered via an application of mobile electronic device 400 forsuperimposing a prerecorded video file over video 404. It should beappreciated that the superimposing of a prerecorded video file over avideo may be performed at a standalone application of mobile electronicdevice 400 or via a plugin/modification of an existing application ofmobile electronic device 400. In one embodiment, the superimposing of aprerecorded video file over video 404 is executed responsive to a userinteraction with control element 410. Control element 410 is operable toactivate/deactivate the superimposing of a prerecorded video file overvideo 404.

Mobile electronic device 400 also plays prerecorded video file 406 overvideo 404. Prerecorded video file 406, as illustrated, includes subjectmatter related to the business location illustrated in video 404. Insome embodiments, prerecorded video file 406 is selected based on thelocation of the scene of video 404. For example, since video 404 is avideo of a local coffee shop, prerecorded video file 406 depicts pouringand preparation of a cup of coffee, so as to entice the user of mobileelectronic device 400 to visit the coffee shop.

As illustrated, video 404 depicts a flat surface 408 within the depictedscene. Flat surface 408 may be, for example, a signage region of thebusiness may be a flat surface or a wall facing may be a flat region.Using depth information, mobile electronic device 400 is operable toposition prerecorded video file 406 such that it appears to beintegrated within flat surface 408. It should be appreciated that insome embodiments, prerecorded video file 406 is positioned withindisplay 402 so as to not obscure important visual elements of video 404(e.g., the name of the business location, the logo of the businesslocation, etc.)

FIG. 4B illustrates an example mobile electronic device 420 having adisplay 422. As illustrated, display 422 is displaying video 424, wherevideo 424 depicts a scene of a business location (e.g., a local coffeeshop). In one embodiment, video 424 is a live video captured by a videocapture device (e.g., a rear-facing camera) of mobile electronic device420. It should be appreciated that in accordance with other embodiments,video 424 may be captured at another mobile electronic device, andtransmitted to mobile electronic device 420 for rendering on display422, either in real-time (e.g., live) or time delayed.

In one embodiment, video 424 is rendered via a camera application ofmobile electronic device 420. In another embodiment, video 424 isrendered via an application of mobile electronic device 420 forsuperimposing a prerecorded video file over video 424. It should beappreciated that the superimposing of a prerecorded video file over avideo may be performed at a standalone application of mobile electronicdevice 420 or via a plugin/modification of an existing application ofmobile electronic device 420. In one embodiment, the superimposing of aprerecorded video file over video 424 is executed responsive to a userinteraction with control element 430. Control element 430 is operable toactivate/deactivate the superimposing of a prerecorded video file overvideo 424.

Mobile electronic device 420 also plays prerecorded video file 426 overvideo 424. Prerecorded video file 426, as illustrated, includes subjectmatter related to the business location illustrated in video 424. Insome embodiments, prerecorded video file 426 is selected based on thelocation of the scene of video 424. For example, since video 424 is avideo of a local coffee shop, prerecorded video file 426 depicts pouringand preparation of a cup of coffee, so as to entice the user of mobileelectronic device 420 to visit the coffee shop.

As illustrated, video 424 depicts a flat surface 428 within the depictedscene. Due to the viewing angle of video 424, flat surface 428 is shownin perspective. Flat surface 428 may be, for example, a signage regionof the business may be a flat surface or a wall facing may be a flatregion. Using depth information, mobile electronic device 420 isoperable to position prerecorded video file 426 in perspective such thatit appears to be integrated within flat surface 428. It should beappreciated that in some embodiments, prerecorded video file 426 ispositioned within display 422 so as to not obscure important visualelements of video 424 (e.g., the name of the business location, the logoof the business location, etc.)

In one embodiment, video 444 is rendered via a camera application ofmobile electronic device 440. In another embodiment, video 444 isrendered via an application of mobile electronic device 440 forsuperimposing a prerecorded video file over video 444. It should beappreciated that the superimposing of a prerecorded video file over avideo may be performed at a standalone application of mobile electronicdevice 440 or via a plugin/modification of an existing application ofmobile electronic device 440. In one embodiment, the superimposing of aprerecorded video file over video 444 is executed responsive to a userinteraction with control element 410. Control element 410 is operable toactivate/deactivate the superimposing of a prerecorded video file overvideo 444.

Mobile electronic device 440 also plays prerecorded video file 446 overvideo 444. Prerecorded video file 446, as illustrated, includes subjectmatter related to the business location illustrated in video 444. Insome embodiments, prerecorded video file 446 is selected based on thelocation of the scene of video 444. For example, since video 444 is avideo of a McDonald's restaurant, prerecorded video file 446 depicts acat sitting atop a McDonald's hamburger floating in space, so as toentice the user of mobile electronic device 440 to visit the McDonald'srestaurant.

As illustrated, video 444 depicts a sign within the depicted scene.Using depth information, mobile electronic device 440 is operable toposition prerecorded video file 446 such that it appears to be affixedto the sign. It should be appreciated that in some embodiments,prerecorded video file 446 is positioned within display 442 so as to notobscure important visual elements of video 444 (e.g., the name of thebusiness location, the logo of the business location, etc.)

In one embodiment, static element 450 is also rendered on display 442.Static element 450 is an element for visually indicating thatprerecorded video file 446 is virtually held in place in space. Forexample, as illustrated, static element 450 depicts a piece of tape,indicating that prerecorded video file 446 is virtually affixed to thesign depicted in video 444. Other examples of static element 450 includea picture frame, a staple, an easel, etc.

In some embodiments, user-generated content may be used to enhanceobjects within scenes. FIGS. 5A through 5C illustrate examples of auser-generated prerecorded video file superimposed over a videopositioned relative to an object within the video, in accordance withvarious embodiments.

FIG. 5A illustrates an example mobile electronic device 500 having adisplay 502. As illustrated, display 502 is displaying video 504, wherevideo 504 depicts a scene of a particular geography (e.g., the Hollywoodsign). In one embodiment, video 504 is a live video captured by a videocapture device (e.g., a rear-facing camera) of mobile electronic device500. It should be appreciated that in accordance with other embodiments,video 504 may be captured at another mobile electronic device, andtransmitted to mobile electronic device 500 for rendering on display502, either in real-time (e.g., live) or time delayed.

In one embodiment, video 504 is rendered via a camera application ofmobile electronic device 500. In another embodiment, video 504 isrendered via an application of mobile electronic device 500 forsuperimposing a prerecorded video file over video 504. It should beappreciated that the superimposing of a prerecorded video file over avideo may be performed at a standalone application of mobile electronicdevice 500 or via a plugin/modification of an existing application ofmobile electronic device 500. In one embodiment, the superimposing of aprerecorded video file over video 504 is executed responsive to a userinteraction with control element 508. Control element 508 is operable toactivate/deactivate the superimposing of a prerecorded video file overvideo 504.

Mobile electronic device 500 also plays prerecorded video file 506 overvideo 504. Prerecorded video file 506, as illustrated, is auser-generated video file that conveys information desired by thecreator that is received and superimposed over video 504 responsive todetermining a location associated with prerecorded video file 506. Inthe illustrated example, prerecorded video file 506 includes a depictionof an airplane dragging a banner including the message “Marry me?” whichmay be specific to a particular user of mobile electronic device 500.Prerecorded video file 506 enhances the experience of the user of mobileelectronic device 500 by conveying a particular message to that user. Itshould be appreciated that in some embodiments, prerecorded video file506 is positioned within display 502 so as to not obscure importantvisual elements of video 504 (e.g., the name of the business location,the logo of the business location, etc.)

FIG. 5B illustrates an example mobile electronic device 520 having adisplay 522. As illustrated, display 522 is displaying video 524, wherevideo 524 depicts a scene of a blank wall in a city environment. In oneembodiment, video 524 is a live video captured by a video capture device(e.g., a rear-facing camera) of mobile electronic device 520. It shouldbe appreciated that in accordance with other embodiments, video 524 maybe captured at another mobile electronic device, and transmitted tomobile electronic device 520 for rendering on display 522, either inreal-time (e.g., live) or time delayed.

In one embodiment, video 524 is rendered via a camera application ofmobile electronic device 520. In another embodiment, video 524 isrendered via an application of mobile electronic device 520 forsuperimposing a prerecorded video file over video 524. It should beappreciated that the superimposing of a prerecorded video file over avideo may be performed at a standalone application of mobile electronicdevice 520 or via a plugin/modification of an existing application ofmobile electronic device 520. In one embodiment, the superimposing of aprerecorded video file over video 524 is executed responsive to a userinteraction with control element 528. Control element 528 is operable toactivate/deactivate the superimposing of a prerecorded video file overvideo 524.

Mobile electronic device 520 also plays prerecorded video file 526 overvideo 524. Prerecorded video file 526, as illustrated, is auser-generated video file that conveys information desired by thecreator. In the illustrated example, prerecorded video file 526 includesa depiction of an ocean scene. Prerecorded video file 526 enhances theexperience of the user of mobile electronic device 520 by conveying aparticular message to that user. The illustrated example allows a userto beautify selected types of surroundings (e.g., blank walls) withvideo enhancements (e.g., waves crashing). For example, a user of mobileelectronic device 520 may subscribe to a feed via a social network thatenhances features of scenes. It should be appreciated that in someembodiments, prerecorded video file 526 is positioned within display 522so as to not obscure important visual elements of video 524 (e.g., thename of the business location, the logo of the business location, etc.)

FIG. 5C illustrates an example mobile electronic device 540 having adisplay 542. As illustrated, display 542 is displaying video 544, wherevideo 544 depicts a scene of a blank wall in a city environment. In oneembodiment, video 544 is a live video captured by a video capture device(e.g., a rear-facing camera) of mobile electronic device 540. It shouldbe appreciated that in accordance with other embodiments, video 544 maybe captured at another mobile electronic device, and transmitted tomobile electronic device 540 for rendering on display 542, either inreal-time (e.g., live) or time delayed.

In one embodiment, video 544 is rendered via a camera application ofmobile electronic device 540. In another embodiment, video 544 isrendered via an application of mobile electronic device 540 forsuperimposing a prerecorded video file over video 544. It should beappreciated that the superimposing of a prerecorded video file over avideo may be performed at a standalone application of mobile electronicdevice 540 or via a plugin/modification of an existing application ofmobile electronic device 540. In one embodiment, the superimposing of aprerecorded video file over video 544 is executed responsive to a userinteraction with control element 548. Control element 548 is operable toactivate/deactivate the superimposing of a prerecorded video file overvideo 544.

Mobile electronic device 540 also plays prerecorded video file 546 overvideo 544. Prerecorded video file 546, as illustrated, is auser-generated video file that conveys information desired by thecreator. In the illustrated example, prerecorded video file 546 includesa depiction of computer-generated molten lava scene that overlays asidewalk. Prerecorded video file 546 enhances the experience of the userof mobile electronic device 540 by conveying a particular message tothat user. The illustrated example allows a user to visually transformselected types of surroundings (e.g., sidewalks, roads, pavement, etc.)with video enhancements (e.g., rushing water, molten lava, etc.). Forexample, a user of mobile electronic device 540 may subscribe to a feedvia a social network that enhances features of scenes. It should beappreciated that in some embodiments, prerecorded video file 546 ispositioned within display 542 so as to not obscure important visualelements of video 544 (e.g., the name of the business location, the logoof the business location, etc.)

Example Methods of Operation

FIG. 6 illustrates a flow diagram 600 of an example method forsuperimposing a prerecorded media file into a video, according tovarious embodiments. Procedures of this method may be described withreference to elements and/or components of various figures describedherein. It is appreciated that in some embodiments, the procedures maybe performed in a different order than described, that some of thedescribed procedures may not be performed, and/or that one or moreadditional procedures to those described may be performed. Flow diagram600 includes some procedures that, in various embodiments, are carriedout by one or more processors under the control of computer-readable andcomputer-executable instructions that are stored on non-transitorycomputer-readable storage media. It is further appreciated that one ormore procedures described in flow diagram 600 may be implemented inhardware, or a combination of hardware with firmware and/or software.

In one embodiment, as shown at procedure 610 of flow diagram 600, livevideo is captured at a video capture device (e.g., optical sensor 135)of a mobile electronic device (e.g., mobile electronic device 100). Atprocedure 620, a video of a scene is displayed on a display device(e.g., display 140) of the mobile electronic device. In one embodiment,the video is live video captured at procedure 610.

In some embodiments, as shown at procedure 630, depth information of thescene captured within the video is determined. The depth informationdefines distances of points of the scene from a video capture device,such that the depth information comprises relative depths of the pointsof the scene based on a position of the video capture device.

At procedure 640, a prerecorded video file to render on the displaydevice is received. At procedure 650, the prerecorded video file issuperimposed over the video, such that the video is partially obscuredby the prerecorded video file. The prerecorded video file is playedwhile displaying the video, such that the prerecorded video file and anon-obscured portion of the video are rendered simultaneously. In someembodiments, the prerecorded video file is selectable for interactivity,wherein a user interaction with the prerecorded video file initiates anaction. (e.g., navigate to a particular website, show content, transformthe prerecorded video file, etc.)

In one embodiment, as shown at procedure 652, the prerecorded video fileis placed over the video at a particular distance from the position ofthe video capture device based on the depth information. In anotherembodiment, as shown at procedure 654, the prerecorded video file isplaced over the video at the distances that the points of the scene arefrom the video capture device based on the depth information, such thatthe prerecorded video file is visually integrated into the scene. In oneembodiment, the depth information includes surface contours of objectswithin the scene, such that the prerecorded video file is placed overthe video following the surface contours of the objects. In anotherembodiment, as shown at procedure 656, a flat region of the scene withinthe video is identified based on the depth information. At procedure658, the prerecorded video file is placed over the video at the flatregion of the scene, such that the prerecorded video file is visuallyintegrated into the scene at the flat region.

At procedure 660, the prerecorded video file is played while displayingthe video, such that the prerecorded video file and a non-obscuredportion of the video are rendered simultaneously.

FIGS. 7A and 7B illustrate a flow diagram 700 of an example method forselecting of a prerecorded media file for superimposing into a video,according to various embodiments. Procedures of this method may bedescribed with reference to elements and/or components of variousfigures described herein. It is appreciated that in some embodiments,the procedures may be performed in a different order than described,that some of the described procedures may not be performed, and/or thatone or more additional procedures to those described may be performed.Flow diagram 700 includes some procedures that, in various embodiments,are carried out by one or more processors under the control ofcomputer-readable and computer-executable instructions that are storedon non-transitory computer-readable storage media. It is furtherappreciated that one or more procedures described in flow diagram 700may be implemented in hardware, or a combination of hardware withfirmware and/or software.

With reference to FIG. 7A, in one embodiment, as shown at procedure 710of flow diagram 700, live video is captured at a video capture device(e.g., optical sensor 135) of a mobile electronic device (e.g., mobileelectronic device 100). At procedure 720, a video of a scene isdisplayed on a display device (e.g., display 140) of the mobileelectronic device. In one embodiment, the video is live video capturedat procedure 710.

At procedure 730, a location of the scene is determined. In oneembodiment, as shown at procedure 732, the location of the scene isdetermined using a navigation satellite system sensor (e.g., locationsensor 170) of the mobile electronic device. In another embodiment, asshown at procedure 734, the location of the scene is determined using athird party application (e.g., Foursquare, Yelp). In another embodiment,as shown at procedure 736, symbol recognition is performed on the videoto identify at least one symbol (e.g., logo, word, phrase, etc.) Atprocedure 738, a location database is searched for the symbol todetermine the location.

In some embodiments, as shown at procedure 740, depth information of thescene captured within the video is determined. The depth informationdefines distances of points of the scene from a video capture device,such that the depth information comprises relative depths of the pointsof the scene based on a position of the video capture device.

At procedure 745, a prerecorded video file to render on the displaydevice is selected based on the location determined at procedure 730. Atprocedure 750, a prerecorded video file to render on the display deviceis received.

With reference to FIG. 7B, at procedure 760, the prerecorded video fileis superimposed over the video, such that the video is partiallyobscured by the prerecorded video file. The prerecorded video file isplayed while displaying the video, such that the prerecorded video fileand a non-obscured portion of the video are rendered simultaneously. Insome embodiments, the prerecorded video file is selectable forinteractivity, wherein a user interaction with the prerecorded videofile initiates an action. (e.g., navigate to a particular website, showcontent, transform the prerecorded video file, etc.)

In one embodiment, as shown at procedure 762, the prerecorded video fileis placed over the video at a particular distance from the position ofthe video capture device based on the depth information. In anotherembodiment, as shown at procedure 764, the prerecorded video file isplaced over the video at the distances that the points of the scene arefrom the video capture device based on the depth information, such thatthe prerecorded video file is visually integrated into the scene. In oneembodiment, the depth information includes surface contours of objectswithin the scene, such that the prerecorded video file is placed overthe video following the surface contours of the objects. In anotherembodiment, as shown at procedure 766, a flat region of the scene withinthe video is identified based on the depth information. At procedure768, the prerecorded video file is placed over the video at the flatregion of the scene, such that the prerecorded video file is visuallyintegrated into the scene at the flat region. In one embodiment, asshown at procedure 770, the prerecorded video file is placed over thevideo without obscuring the at least one symbol.

At procedure 780, the prerecorded video file is played while displayingthe video, such that the prerecorded video file and a non-obscuredportion of the video are rendered simultaneously.

Example Network and System for Tracking Interactivity with PrerecordedMedia Files

Directing communications to interested or potentially interested partiesis typically a challenging endeavor. Whether the initiating party is abusiness (e.g., for-profit or non-profit), a governmental agency, or aservice provider, it is often difficult to effectively target a desiredaudience of a communication. For example, billboards and televisionadvertisements, while subject to some focusing such as a high levelmakeup of the viewing population, are generally scattershot approachesto communicating. The advent of the Internet has improved communicationtargeting, but is still limited in the types of information that can bepresented and when to present certain types of communications.

Embodiments described herein improve the targeting of communication topeople such that a communication (e.g., an advertisement ornotification) can be presented to people at an appropriate place andtime to increase effectiveness of the communication while integratingthe communication into a person's view of the world. Presentingcommunications to people in a contextual and palatable manner increasesthe effectiveness of the communication, reduces overuse and/orsaturation of the communication, and improves the return on investmentof the communicating party. In other words, presenting a communicationto a person at a time and location, and in a contextual and engagingmanner, increases the efficacy of the communication.

The described embodiments improve the technology of communicatinginformation by integrating prerecorded media files into the real worldwhile engaging with the real world. Performance of the computer systemis improved by not unnecessarily presenting communications in a mannerthat is less effective, thereby not wasteful of processing and networkbandwidth, power consumption of mobile devices, time of the viewingparty, and financial resources of the presenting party.

In accordance with various embodiments, a prerecorded media file may bepresented for display according to the instructions of a placing party(e.g., a business or an advertiser) within a live video of the realworld in which the viewing party is located. For instance, a businessmay want to display a prerecorded video file on a display screen of amobile electronic device when the mobile electronic device isgeographically proximate a location of the business. For example,Starbucks may want to display a prerecorded video file when the mobileelectronic device is near a Starbucks location. The placing party canprovide presentation instructions that dictate conditions for displayingthe prerecorded video file, expiration conditions that dictate how usageof a prerecorded video file is terminated, and/or user-based conditionsthat dictate conditions for displaying a prerecorded video file.Moreover, in accordance with various embodiments, instances of thedisplay of a prerecorded video file and instances of interaction with aprerecorded video file can be logged for communicating to the placingparty.

A mobile electronic device (e.g., mobile electronic device 100) displaysvideo of a scene on a display device (e.g., display 140). In oneembodiment, the video is live video of a scene captured by an opticalsensor of the mobile electronic device (e.g., a rear-facing camera). Insuch an embodiment, the scene is viewable to a user of the mobileelectronic device both on the display device of the mobile electronicdevice and without using the display device. As the user moves throughthe world, the mobile electronic device can display prerecorded videofiles superimposed into the scene viewable through the display device.The prerecorded video files are integrated into the live scene of thereal world, enhancing the viewing experience of the user.

In some embodiments, prerecorded media files are displayed on a mobileelectronic device, superimposed on a live video, according topresentation instructions. FIG. 8 illustrates another embodiment ofexample communication network 240 upon which embodiments describedherein may be implemented. FIG. 8 illustrates electronic device 210,remote computer system 230, and remote computer system 810, which arecommunicatively coupled via network 240. It should be appreciated thatelectronic device 210 may be implemented as a mobile electronic device100, and/or include any combination of the components of mobileelectronic device 100.

Electronic device 210 is capable of displaying video of a scene (e.g.,e.g., scene 250 at display 140). In one embodiment, the scene iscaptured at a video capture device (e.g., optical sensor 135-1) ofelectronic device 210. Electronic device 210 may be associated with aparticular user. Embodiments of the operation remote computer system 230and electronic device 210 are described above with reference to FIG. 2.

Remote computer system 810 includes prerecorded media file 820 andpresentation instructions 830. Prerecorded media file 820 can be anytype of file that can be rendered on an electronic device 210 (e.g., aprerecorded video file). It should be appreciated that any type of mediafile format can be used in accordance with the described embodiments,including but not limited to Graphics Interchange Format (GIF), WebM,WebP, MPEG-4 (MP4), Animated Portable Network Graphics (APNG), MotionJPEG, Flash video (FLV), Windows Media video, M4V, etc. It should beappreciated that the prerecorded media file can be looped (e.g., via aHTML 5 video element or Flash video element) to automatically repeat. Italso should be appreciated that remote computer system 810 can includeany number of prerecorded media files 820 and presentation instructions830.

Presentation instructions 830 are associated with prerecorded media file820, and include display conditions for displaying prerecorded mediafile 820 on electronic device 210. The display conditions are conditionsdictating when and how prerecorded media file 820 is displayed onelectronic device 210. In one embodiment, the display conditions ofpresentation instructions 830 include one or more geographic locationsfor displaying prerecorded media file 820 on electronic device 210. Forexample, responsive to detecting that electronic device 210 is at ageographic location or proximate a geographic location (e.g., within aparticular distance of a geographic location as defined in the displayconditions), prerecorded media file 820 is displayed on electronicdevice 210. In such an example, a user may be walking around town andviewing video of a live scene of their environment through anapplication operating on electronic device 210 (e.g., captured via arear-facing camera). Responsive to detecting that electronic device 210is at or proximate a particular business, prerecorded media file 820 isdisplayed superimposed over the video of the live scene. In a particularexample, the presentation instructions 830 include display conditions todisplay prerecorded media file 820 on electronic device 210 responsiveto electronic device being at or proximate a McDonalds restaurant, wherethe content of the prerecorded media file 820 is related to a McDonaldsdining experience.

In other embodiments, the display conditions include identifying alocation. In one embodiment, the location of the scene is determinedusing a location sensor (e.g., location sensor 170) of the mobileelectronic device. In another embodiment, the location of the scene isdetermined using a third party application (e.g., Foursquare, Yelp). Inanother embodiment, symbol recognition is performed on the video toidentify at least one symbol (e.g., logo, word, phrase, etc.) usinglocation database 234.

In some embodiments, the display conditions of presentation instructions830 include temporal conditions. For example, a display condition mayinclude a condition that prerecorded media file 820 is only availablefor display during particular hours of the day. In one example, thedisplay conditions may dictate that prerecorded media file is onlyavailable for display during business hours of the associated business.

In some embodiments, the display conditions of presentation instructions830 include geo-temporal conditions. Geo-temporal conditions are acombination of location conditions and temporal conditions. For example,a geo-temporal conditions may dictate that a prerecorded media file 820is displayed in response to detecting that electronic device 210 is ator proximate a particular location at a particular time or within aparticular temporal range (e.g., 8:00 AM through 10:00 PM). In oneexample, remote computer system 810 may include multiple prerecordedmedia files 820 for a business, such that a first prerecorded media fileis available for display during business hours and a second prerecordedmedia file is available for display during hours where the business isclosed.

In some embodiments, presentation instructions 830 include expirationconditions for terminating usage of prerecorded media file 820 atelectronic device 210. In order to increase effectiveness of prerecordedmedia file 820, the party responsible for placing prerecorded media file820 according to presentation instructions 830 may desire to control thenumber of times that prerecorded media file 820 is displayed at aparticular electronic device 210 and/or to a particular user. In oneembodiment, the expiration condition is a number of times prerecordedmedia file 820 is displayed (e.g., one time, two times, etc.) In anotherembodiment, the expiration condition is a frequency at which prerecordedmedia file 820 can be displayed (e.g., no more than once per every fourhours, no more than once per day, etc.) In another embodiment, theexpiration condition is a date and/or time (e.g., expires on December31, expires at 1:00 PM, etc.) In another embodiment, the expirationcondition is a number of user interactions with prerecorded media file820 (e.g., one interaction, two interactions, etc.) In anotherembodiment, prerecorded media file 820 expires upon detecting thatelectronic device 210 is no longer at or proximate a location. It shouldbe appreciated that other expiration conditions may be used. The use ofexpiration conditions allows for the party placing prerecorded mediafile 820 to granularly control presentation of prerecorded media file820, limiting overuse to reduce user exhaustion, as well as reducingprocessing usage, bandwidth usage, and power consumption of electronicdevice 210.

In some embodiments, presentation instructions 830 include user-basedconditions for displaying prerecorded media file 820. The user-basedconditions may be based on a user profile for a user operatingelectronic device 210, where the user profile may include explicitinformation provided by the user (e.g., age, gender, interests, etc.),implicit information gleaned from the user's actions (e.g., internetbrowsing history, frequented locations, etc.), or any combinationthereof. Moreover, the user profile may be received from a third-party(e.g., Facebook, Instagram, etc.) In order to increase effectiveness ofprerecorded media file 820, the party responsible for placingprerecorded media file 820 according to presentation instructions 830may target particular users based on particular user-based conditions.For example, the placing party may want prerecorded media file 820 usersof a particular gender and/or age range to see a particular prerecordedmedia file 820. In one example, a business may be a clothing store forwoman in their twenties and thirties. Presentation instructions 830could include user-based conditions to only display this prerecordedmedia file 820 to female users with ages twenty through thirty-nine. Theuse of user-based conditions further allows for the party placingprerecorded media file 820 to granularly control presentation ofprerecorded media file 820, limiting overuse to reduce user exhaustion,as well as reducing processing usage, bandwidth usage, and powerconsumption of electronic device 210.

In order to evaluate effectiveness of a particular prerecorded mediafile 820, in some embodiments, electronic device 210 maintains usage log840. Responsive to a prerecorded media file 820 being displayed onelectronic device 210, the display instance is logged to usage log 840.The placing party can access usage log 840 for a particular prerecordedmedia file 820, and use this information to assist in evaluatingeffectiveness of the prerecorded media file 820.

In some embodiments, the prerecorded media file 820 is selectable forinteractivity. For example, a user may interact with prerecorded mediafile 820 (e.g., touching it via a touch screen) while it is displayed onelectronic device 210 to execute an action. For example, a userinteraction with prerecorded media file 820 may open up an Internetbrowser application and navigate to a particular website. In otherexamples, a user interaction with prerecorded media file 820 may showcontent (e.g., a coupon or product information). In some embodiments,prerecorded media file 820 may be transformed as a result of a userinteraction (e.g., the prerecorded video file can be resized or movedwithin a display).

In order to further evaluate effectiveness of a particular prerecordedmedia file 820, in some embodiments, interaction instances withprerecorded media file 820 can also be logged to usage log 840.Responsive to an interaction with (e.g., selection of) a prerecordedmedia file 820 being displayed on electronic device 210, the interactioninstance is logged to usage log 840. The placing party can access usagelog 840 including interaction instances for a particular prerecordedmedia file 820, and use this information to assist in evaluatingeffectiveness of the prerecorded media file 820.

For example, as a user moves through the world, electronic device 210detects satisfaction of display conditions (e.g., geo-temporal) and/oruser-based conditions of presentation instructions 830 and displays aparticular prerecorded media file 820 associated with a location, wherethe location is a Gap clothing store. The user, having been targeted bya party placing prerecorded media file 820, is presented withprerecorded media file 820 that shows a looping short form video of aGap advertisement. The user can then interact (e.g., select) prerecordedmedia file 820 while it is being displayed, and is presented with acoupon for use at the nearby Gap clothing store location. The display ofprerecorded media file 820 and the user interaction with prerecordedmedia file 820 are logged as a display instance and interaction instancein usage log 840.

FIG. 9 illustrates a flow diagram 900 of an example method for trackinginteractivity with a prerecorded media file, according to variousembodiments. Procedures of this method may be described with referenceto elements and/or components of various figures described herein. It isappreciated that in some embodiments, the procedures may be performed ina different order than described, that some of the described proceduresmay not be performed, and/or that one or more additional procedures tothose described may be performed. Flow diagram 900 includes someprocedures that, in various embodiments, are carried out by one or moreprocessors under the control of computer-readable andcomputer-executable instructions that are stored on non-transitorycomputer-readable storage media. It is further appreciated that one ormore procedures described in flow diagram 900 may be implemented inhardware, or a combination of hardware with firmware and/or software.

In one embodiment, as shown at procedure 905 of flow diagram 900, aprerecorded video file to render on the display device is received. Inone embodiment, the prerecorded video file is a looping video file. Inone embodiment, the prerecorded video file is selectable forinteractivity, wherein a user interaction with the displayed prerecordedvideo file initiates an action. In accordance with various embodiments,the action includes, without limitation, opening a web browser todisplay a web page, displaying a coupon, or opening an application.

At procedure 910, presentation instructions for displaying theprerecorded video file on a display device (e.g., display 140) of amobile electronic device (e.g., mobile electronic device 100) arereceived, where the presentation instructions include display conditionsfor displaying the prerecorded video file. In one embodiment, thedisplay conditions include detection of a geographic location of themobile electronic device. In one embodiment, the display conditionsinclude one of temporal conditions, geographic conditions, andgeo-temporal conditions. In one embodiment, the presentationinstructions further include expiration conditions for terminating usageof the prerecorded video file at the mobile electronic device. In oneembodiment, the expiration conditions include one of: a number of timesthe prerecorded video file is displayed, a time period after an initialdisplay of the prerecorded video file, and a number of user interactionswith the prerecorded video file. In one embodiment, the presentationinstructions further comprise user-based conditions for displaying theprerecorded video file.

At procedure 920, a video of a scene is displayed on the display deviceof the mobile electronic device. In one embodiment, the video is livevideo captured at a video capture device (e.g., optical sensor 135) ofthe mobile electronic device.

At procedure 930, responsive to detecting at least one display conditionof the display conditions, the prerecorded video file is displayed onthe display device of the mobile electronic device such that the videois partially obscured by the prerecorded video file. At procedure 940,responsive to the displaying the prerecorded video file, a displayinstance for the prerecorded video file is logged.

In one embodiment, as shown at procedure 950, responsive to detecting auser interaction with the displayed prerecorded video file, the actionis initiated. In one embodiment, as shown at procedure 960, responsiveto detecting a user interaction with the displayed prerecorded videofile, an interaction instance for the prerecorded video file is logged.

CONCLUSION

The examples set forth herein were presented in order to best explain,to describe particular applications, and to thereby enable those skilledin the art to make and use embodiments of the described examples.However, those skilled in the art will recognize that the foregoingdescription and examples have been presented for the purposes ofillustration and example only. Many aspects of the different exampleembodiments that are described above can be combined into newembodiments. The description as set forth is not intended to beexhaustive or to limit the embodiments to the precise form disclosed.Rather, the specific features and acts described above are disclosed asexample forms of implementing the claims.

Reference throughout this document to “one embodiment,” “certainembodiments,” “an embodiment,” “various embodiments,” “someembodiments,” or similar term means that a particular feature,structure, or characteristic described in connection with the embodimentis included in at least one embodiment. Thus, the appearances of suchphrases in various places throughout this specification are notnecessarily all referring to the same embodiment. Furthermore, theparticular features, structures, or characteristics of any embodimentmay be combined in any suitable manner with one or more other features,structures, or characteristics of one or more other embodiments withoutlimitation.

What is claimed is:
 1. A mobile electronic device comprising: a videocapture device; a display device; a data storage unit; and a processorcoupled with the data storage unit and the display device, the processorconfigured to: receive a prerecorded video file at the mobile electronicdevice, wherein the prerecorded video file is a looping video file,wherein the prerecorded video file is selectable for interactivity, andwherein a user interaction with a displayed prerecorded video fileinitiates an action; receive presentation instructions for displayingthe prerecorded video file on the display device, the presentationinstructions comprising display conditions for displaying theprerecorded video file; display a video of a scene on the display deviceof the mobile electronic device, wherein the video is captured at thevideo capture device; responsive to detecting at least one displaycondition of the display conditions, display the prerecorded video fileon the display device of the mobile electronic device, such that thevideo is partially obscured by the prerecorded video file; andresponsive to the displaying the prerecorded video file, log a displayinstance for the prerecorded video file.
 2. The mobile electronic deviceof claim 1, wherein the processor is further configured to: responsiveto detecting a user interaction with the displayed prerecorded videofile, initiate the action.
 3. The mobile electronic device of claim 2,wherein the processor is further configured to: responsive to detectinga user interaction with the displayed prerecorded video file, log aninteraction instance for the prerecorded video file.
 4. The mobileelectronic device of claim 1, wherein the display conditions comprisedetection of a geographic location of the mobile electronic device. 5.The mobile electronic device of claim 1, wherein the display conditionscomprise one of temporal conditions, geographic conditions, andgeo-temporal conditions.
 6. The mobile electronic device of claim 1,wherein the presentation instructions further comprise expirationconditions for terminating usage of the prerecorded video file at themobile electronic device.
 7. The mobile electronic device of claim 6,wherein the expiration conditions comprise one of: a number of times theprerecorded video file is displayed, a time period after an initialdisplay of the prerecorded video file, and a number of user interactionswith the prerecorded video file.
 8. The mobile electronic device ofclaim 1, wherein the presentation instructions further compriseuser-based conditions for displaying the prerecorded video file.
 9. Amethod for tracking interactivity with a prerecorded video filesuperimposed into a video, the method comprising: receiving presentationinstructions for displaying a prerecorded video file on a display deviceof a mobile electronic device, the presentation instructions comprisingdisplay conditions for displaying the prerecorded video file; displayinga video of a scene on the display device of the mobile electronicdevice; responsive to detecting at least one display condition of thedisplay conditions, displaying the prerecorded video file on the displaydevice of the mobile electronic device, such that the video is partiallyobscured by the prerecorded video file; and responsive to the displayingthe prerecorded video file, logging a display instance for theprerecorded video file.
 10. The method of claim 9, wherein theprerecorded video file is selectable for interactivity, wherein a userinteraction with the displayed prerecorded video file initiates anaction.
 11. The method of claim 10, further comprising: responsive todetecting a user interaction with the displayed prerecorded video file,initiating the action.
 12. The method of claim 10, further comprising:responsive to detecting a user interaction with the displayedprerecorded video file, logging an interaction instance for theprerecorded video file.
 13. The method of claim 9, wherein the displayconditions comprise detection of a geographic location of the mobileelectronic device.
 14. The method of claim 9, wherein the displayconditions comprise one of temporal conditions, geographic conditions,and geo-temporal conditions.
 15. The method of claim 9, wherein thepresentation instructions further comprise expiration conditions forterminating usage of the prerecorded video file at the mobile electronicdevice.
 16. The method of claim 15, wherein the expiration conditionscomprise one of: a number of times the prerecorded video file isdisplayed, a time period after an initial display of the prerecordedvideo file, and a number of user interactions with the prerecorded videofile.
 17. The method of claim 9, wherein the presentation instructionsfurther comprise user-based conditions for displaying the prerecordedvideo file.
 18. The method of claim 9, further comprising: receiving theprerecorded video file at the mobile electronic device.
 19. The methodof claim 9, wherein the prerecorded video file is a looping video file.20. A non-transitory computer readable storage medium having computerreadable program code stored thereon for causing a computer system toperform a method for tracking interactivity with a prerecorded mediafile, the method comprising: receiving the prerecorded video file at themobile electronic device, wherein the prerecorded video file is alooping video file, wherein the prerecorded video file is selectable forinteractivity, and wherein a user interaction with the displayedprerecorded video file initiates an action; receiving presentationinstructions for displaying a prerecorded video file on a display deviceof a mobile electronic device, the presentation instructions comprisingdisplay conditions for displaying the prerecorded video file andexpiration conditions for terminating usage of the prerecorded videofile at the mobile electronic device, wherein the display conditionscomprise detection of a geographic location of the mobile electronicdevice; displaying a video of a scene on the display device of themobile electronic device; responsive to detecting at least one displaycondition of the display conditions, displaying the prerecorded videofile on the display device of the mobile electronic device, such thatthe video is partially obscured by the prerecorded video file;responsive to the displaying the prerecorded video file, logging adisplay instance for the prerecorded video file; responsive to detectinga user interaction with the displayed prerecorded video file, initiatingthe action; and responsive to detecting a user interaction with thedisplayed prerecorded video file, logging an interaction instance forthe prerecorded video file.